Free Energy Minimization Using the 2-D Cluster Variation Method: Initial Code Verification and Validation

A new approach for general artificial intelligence (GAI), building on neural network deep learning architectures, can make use of one or more hidden layers that have the ability to continuously reach a free energy minimum even after input stimulus is removed, allowing for a variety of possible behaviors. One reason that this approach has not been developed until now has been the lack of a suitable free energy equation; one that would avoid some of the difficulties known in Hopfield-style neural networks. The Cluster Variation Method (CVM) offers a means for characterizing 2-D local pattern distributions, or configuration variables, and provides a free energy formalism in terms of these configuration variables. The equilibrium distribution of these configuration variables is defined in terms of a single interaction enthalpy parameter, h, for the case of equiprobable distribution of bistate (neural/neural ensemble) units. For non-equiprobable distributions, the equilibrium distribution can be characterized by providing a fixed value for the fraction of units in the active state (x1), corresponding to the influence of a per-unit activation enthalpy, together with the pairwise interaction enthalpy parameter h. This paper provides verification and validation (V&V) for code that computes the configuration variable and thermodynamic values for 2-D CVM grids characterized by different interaction enthalpy parameters, or h-values. This means that there is now a working foundation for experimenting with a 2-D CVM-based hidden layer that can, as an alternative to responding strictly to inputs, also now independently come to its own free energy minimum. Such a system can also return to a free energy-minimized state after it has been perturbed, which will enable a range of input-independent behaviors that have not been hitherto available. A further use of this 2-D CVM grid is that by characterizing different kinds of patterns in terms of their corresponding h-values (together with their respective fraction of active-state units), we have a means for quantitatively characterizing different kinds of neural topographies. This further allows us to connect topographic descriptions (in terms of local patterns) with free energy minimization, allowing a first-principles approach to characterizing topographies and building new computational engines.